Who ’ s # 1 ? The Science Of Building Ranking Systems

Sports ranking models were the main area of research for our REU project. We studied several different models and the theory behind them. Utilizing MATLAB as a programming language, we used these models to predict game outcomes and point spreads, and compare the relative value of statistics as indicators of team strength. Keener’s Direct Method (1993) ranking model was the starting point in our research. In order to fully understand Keener’s model we studied the Perron-Frobenius Theorem, which provides the theoretical basis for Keener’s ranking system. We then examined Anjela Govan’s Markov model (2008) and Offense-Defense model (2008). The Markov model is based on the theory of Markov chains and the Offense-Defense model is an application of the Sinkhorn-Knopp Theorem (1967). Finally, we implemented the Massey model (1997), which relies on the theory of least squares and is the basis for one of the computer models used in the BCS. After implementing each of the models, we used NFL data from the 2002-2008 seasons to make game predictions. We compared their efficacy in terms of hindsight and foresight prediction accuracy. In hindsight prediction, we used the ranking generated from an entire season of data to predict all the games from that season. Foresight predication refers to predicting a week of games using the ranking from previous weeks. The models performed similarly in both hindsight and foresight game prediction. Next, we attempted to use the models to predict the point spread of games. We plotted point spreads against the rating differences in the models. Lacking a clear trend or any further ideas, however, we decided that our time would be better spent on other questions. One of these questions was whether we could determine the maximum possible hindsight accuracy for a season. In doing so, we realized that higher hindsight accuracy did not necessarily correspond to higher foresight accuracy, which we consider a better measure of a model’s strength. We were able to generate hindsight accuracies significantly higher than those from the previously mentioned models, but abandoned this idea due to disappointing foresight results. The area on which we spent the most time and made the most progress was our attempt to determine whether rushing or passing yards is a superior game predictor and indicator of team strength. To explore this problem, we first applied each of the four models to rushing yards and passing yards. We also ran the Markov model with combinations of scores with rushing yards and scores with passing yards. Each application produced a higher foresight accuracy using rushing yards than passing yards. To strengthen our results, we also explored the correlation of differences in game